The ‘overview’ and ‘guide’ has been developed primarily for on audience, where users are familiar with wikis have an professional educational background, but do not have much experience in evaluating training programs. The ‘guide’ uses a simple and methodical, step-by-step, colour coded approach, covering the whole evaluation from start to finish.
The information provided, is based on well-known sources, with an initial template integrating methods and theories from Taylor Powell (1996), JISC (2012) and Reeves (2009), informing the outline of questions to consider within the evaluation, with additional theoretical methodologies from Harvey, Fretchling (2002) and Vavoula, G., & Sharples, M. (2009).
The strengths of this ‘guide’, is that it is easy to use and is comprehensive in structure. It has been designed with an ‘overview’ that is planned to be read as a primary interface, with ‘the guide’ as a more practical interactive platform, requiring decision making processes, and practical input strategies. This is a real strength of the guide as it is a step-by-step walk through a self-contained and easily understood environment providing a thorough outline of ideas already considered or ones that need to be incorporated. Both the ‘guide’ and the ‘overview’ can be used independently of each other depending on whether the evaluators simply want to refresh their knowledge or just want a tool for the evaluation itself.
Mobile learning is difficult to evaluate with few specific resources available. However, a series of good instruments are provided and suggested, as well as a clear process to implement the methodology and additional articles, tools, videos, the initial template and references to achieve the particular evaluation goal.
In terms of weaknesses, as comprehensive as the ‘guide’ aims to be there could be more information about the specific influence of mobile technology and its engagements with stakeholders and audiences. Additional approaches, examples and tips throughout each section of the ‘guide’ would be valuable.
There is a distinct dichotomy as too how much information is enough and how much is distracting. Within the framework of the ‘guide’ it has been a significant challenge to present a useful, appropriate interface that is applicable to a variety of mobile technologies, learning environment, contexts and evaluators experience levels and not extend the guide so it becomes unmanageable.
More specific guidance and examples of data and information needed could have being provided e.g. gaining permission, management of an evaluation, methods and so forth.
Possible Improvements Given additional time, it would have been ideal to show examples of these tools put into action as well as providing the user of this guide a look at potentially barriers to success for implementing. This could be provided on a separate page within the wiki.
Currently the system is a static environment with tables suggesting outlines of how information could be collated. A more interactive interface would be useful, with questions and decisions entered directly into the various sub sectioned tables, i.e. ‘5.6 How will the evaluation be communicated and shared?’, that could be automatically collated into 6.1 Evaluation Plan Summary, providing live updates and a fully interactive plan.
5.6 How will the evaluation be communicated and shared? (Interactive interface)
automatically collated
6.1 Evaluation Plan Summary
Mandy
(614 words)
Creating the wiki was an effective way of capturing the plethora of artefacts, information, tools and instruments, especially in the first instance as we embarked on this project. The guide is a rich source of interventions, suggestions, thought provoking questions and points to consider in succinctly being able to define and build a robust evaluation plan for mobile learning.
In our focus of developing this resource for mobile learning evaluation we remained true to the cause. Although there is much common ground in the evaluation space there is also difference which was highlighted, especially in step 6 of 'the overview' and with 'the guide'. Mobile learning evaluation is distinct because of the range of educational and informal settings, the scope of portable technologies that are constantly being up-dated and the triad that has emerged of not only about settings, the learner, the educator now has the capacity to be mobile.
The flow of developing an evaluation plan is in the order of the wiki;
home page - an explanation of how the wiki was balanced and interfaced to meet the assignment needs and plan development
the overview - which is the 'how to' of using the guide
the guide - this is the mobile learning evaluation workbook, which by following a plan will evolve
the template - underpinned theories that interfaced effectively to create a template as a useful instrument to align with the guide in developing an evaluation plan
the tools - this was a vehicle for placing all the useful artifacts we found and knew of, along with other information we wanted to include that didn't belong in anywhere else we felt
the videos - were easy to locate and value added visual information
the articles - a place to have copies of the information and research used to inform, influence and create this wiki and assignment
the reference - authenticating our research and information
the bios - outline of the developers of this wiki and assignment authors
the comments - as time goes by people may use this guide.
Vavoula and Sharples (2009) discuss six challenges in evaluating mobile learning which have a parallel process in developing this assignment.
Challenge 1: Capturing learning context and learning across contexts: This was in devising 'tools and methods appropriate for capturing and analysing mobile learning contexts'. The wiki is a rich resource of tools and methods.
Challenge 2: Has anyone learning anything? In developing a guide and instructions on how to use the guide specifically for mobile learning was certainly a learning experience. Receiving comments of others application of the guide will inform us of learning outcomes.
Challenge 3: An ethical question: Addressed in this evaluation process by acknowledging the need for the least amount of disruption and qualified evaluators. However wonder if there could have been further explicit detail about the level of ethical process i.e. confidentiality, permission to use information and privacy.
Challenge 4: Mobile technology: The wiki itself was challenging in that the links were not fully functioning and navigating easily around in the editing functionality and pages of the wiki were cumbersome at times. As an e-collection it was effective.
Challenge 5: Seeing the bigger picture: The wiki allowed us to present the bigger picture in one medium with many areas to attain knowledge.
Challenge 6: Formal or informal: Both of these areas of learning were addressed in the overview and in the guide. The parallel process to this could be that this resource could be used both informally or formally.
In summary, this has been an integrative learning experience applying theoretical and underpinning knowledge to a practical task for application.
| Alison | MandyAlison
(Word Count 519)The ‘overview’ and ‘guide’ has been developed primarily for on audience, where users are familiar with wikis have an professional
educational background, but do not have much experience in evaluating training programs. The ‘guide’ uses a simple and methodical, step-by-step, colour coded approach, covering the whole evaluation from start to finish.
The information provided, is based on well-known sources, with an initial template integrating methods and theories from Taylor Powell (1996), JISC (2012) and Reeves (2009), informing the outline of questions to consider within the evaluation, with additional theoretical methodologies from Harvey, Fretchling (2002) and Vavoula, G., & Sharples, M. (2009).
The strengths of this ‘guide’, is that it is easy to use and is comprehensive in structure. It has been designed with an ‘overview’ that is planned to be read as a primary interface, with ‘the guide’ as a more practical interactive platform, requiring decision making processes, and practical input strategies. This is a real strength of the guide as it is a step-by-step walk through a self-contained and easily understood environment providing a thorough outline of ideas already considered or ones that need to be incorporated. Both the ‘guide’ and the ‘overview’ can be used independently of each other depending on whether the evaluators simply want to refresh their knowledge or just want a tool for the evaluation itself.
Mobile learning is difficult to evaluate with few specific resources available. However, a series of good instruments are provided and suggested, as well as a clear process to implement the methodology and additional articles, tools, videos, the initial template and references to achieve the particular evaluation goal.
In terms of weaknesses, as comprehensive as the ‘guide’ aims to be there could be more information about the specific influence of mobile technology and its engagements with stakeholders and audiences. Additional approaches, examples and tips throughout each section of the ‘guide’ would be valuable.
There is a distinct dichotomy as too how much information is enough and how much is distracting. Within the framework of the ‘guide’ it has been a significant challenge to present a useful, appropriate interface that is applicable to a variety of mobile technologies, learning environment, contexts and evaluators experience levels and not extend the guide so it becomes unmanageable.
More specific guidance and examples of data and information needed could have being provided e.g. gaining permission, management of an evaluation, methods and so forth.
Possible Improvements
Given additional time, it would have been ideal to show examples of these tools put into action as well as providing the user of this guide a look at potentially barriers to success for implementing. This could be provided on a separate page within the wiki.
Currently the system is a static environment with tables suggesting outlines of how information could be collated. A more interactive interface would be useful, with questions and decisions entered directly into the various sub sectioned tables, i.e. ‘5.6 How will the evaluation be communicated and shared?’, that could be automatically collated into 6.1 Evaluation Plan Summary, providing live updates and a fully interactive plan.
Mandy
(614 words)
Creating the wiki was an effective way of capturing the plethora of artefacts, information, tools and instruments, especially in the first instance as we embarked on this project. The guide is a rich source of interventions, suggestions, thought provoking questions and points to consider in succinctly being able to define and build a robust evaluation plan for mobile learning.
In our focus of developing this resource for mobile learning evaluation we remained true to the cause. Although there is much common ground in the evaluation space there is also difference which was highlighted, especially in step 6 of 'the overview' and with 'the guide'. Mobile learning evaluation is distinct because of the range of educational and informal settings, the scope of portable technologies that are constantly being up-dated and the triad that has emerged of not only about settings, the learner, the educator now has the capacity to be mobile.
The flow of developing an evaluation plan is in the order of the wiki;
Vavoula and Sharples (2009) discuss six challenges in evaluating mobile learning which have a parallel process in developing this assignment.
Challenge 1: Capturing learning context and learning across contexts: This was in devising 'tools and methods appropriate for capturing and analysing mobile learning contexts'. The wiki is a rich resource of tools and methods.
Challenge 2: Has anyone learning anything? In developing a guide and instructions on how to use the guide specifically for mobile learning was certainly a learning experience. Receiving comments of others application of the guide will inform us of learning outcomes.
Challenge 3: An ethical question: Addressed in this evaluation process by acknowledging the need for the least amount of disruption and qualified evaluators. However wonder if there could have been further explicit detail about the level of ethical process i.e. confidentiality, permission to use information and privacy.
Challenge 4: Mobile technology: The wiki itself was challenging in that the links were not fully functioning and navigating easily around in the editing functionality and pages of the wiki were cumbersome at times. As an e-collection it was effective.
Challenge 5: Seeing the bigger picture: The wiki allowed us to present the bigger picture in one medium with many areas to attain knowledge.
Challenge 6: Formal or informal: Both of these areas of learning were addressed in the overview and in the guide. The parallel process to this could be that this resource could be used both informally or formally.
In summary, this has been an integrative learning experience applying theoretical and underpinning knowledge to a practical task for application.